Multivariate Density Estimation with Deep Neural Mixture Models

Abstract Albeit worryingly underrated in the recent literature on machine learning general (and, deep particular), multivariate density estimation is a fundamental task many applications, at least implicitly, and still an open issue. With few exceptions, neural networks (DNNs) have seldom been applied to estimation, mostly due unsupervised nature of task, (especially) need for constrained training algorithms that ended up realizing proper probabilistic models satisfy Kolmogorov’s axioms. Moreover, spite well-known improvement terms modeling capabilities yielded by mixture over plain single-density statistical estimators, no mixtures DNN-based component densities investigated so far. The paper fills this gap extending our previous work (NMMs) DNN mixtures. A maximum-likelihood (ML) algorithm estimating Deep NMMs (DNMMs) handed out, which satisfies numerically combination hard soft constraints aimed ensuring satisfaction class probability functions can be modeled any degree precision via DNMMs formally defined. procedure automatic selection DNMM architecture, as well hyperparameters its ML algorithm, presented (exploiting DNMM). Experimental results univariate data are reported on, corroborating effectiveness approach superiority most popular techniques.